Measuring Perceptions and Preferences for Meritocracy
Juan Carlos Castillo, Julio Iturra, Luis Maldonado, Francisco Meneses & Jorge Atria
Abstract
Economic and social inequalities have generated growing concern and crises across contemporary societies. One of the mechanisms proposed by social sciences to explain the persistence of inequality is the belief in meritocracy, which would legitimize economic disparities based on differences in effort and talent. Despite its wide use as a concept, empirical research on meritocracy is relatively novel and characterized by diverse conceptualization and measures that make the findings and their interpretation rather confusing. Most of the studies in the area have relied upon secondary data to operationalize meritocracy, with a wide variation in the use and interpretation of the same survey items. Taking into account the extant literature that uses measures of meritocracy, this article identifies a series of drawbacks and inconsistencies within and between studies regarding conceptualization and operationalization. Based on this critical analysis, we propose a survey instrument called Perceptions and Preferences for Meritocracy Scale, which we test with confirmatory factor analysis using data from an online survey study (N=2,141). Our results support the proposed conceptual structure, which not only distinguishes between perceptions and preferences, but also between meritocratic and non-meritocratic dimensions. Our discussion highlights the importance of considering these different dimensions in order to advance the study of meritocracy.Introduction
Economic inequality has become an issue of growing concern around the world. This has been expressed in a series of protests, as well as diverse critical analyses regarding the development of capitalism and its consequences (streeckPoliticsPublicDebt2014?; piketty_Capital_2014?). In this context, the study of views, preferences and perceptions of inequality has acquired relevance in the social sciences, in topics such as redistributive preferences (alesina_Fairness_2005?; dimick_Models_2018?), the legitimization of economic inequality (schroder_Income_2017?) and the functioning of meritocracy (duru-bellat_whos_2012?; Mijs 2019; Reynolds and Xian 2014; atriaEconomicElitesAttitudes2020?).
In general, meritocracy is defined as a system of distribution of resources and rewards based on individual merit, which in its original conception is a combination of talent and effort (M. Young 1962). This traditional conception of merit places in a secondary position the possible interference of structural or non-meritocratic factors, such as inheritance, personal contacts, and luck (Breen and Goldthorpe 1999; Saunders 1995; Yair 2007; Land 2006; M. D. Young 1994). Social psychology and sociology have studied the characteristics and consequences of beliefs in meritocracy, under the general hypothesis that a greater belief in meritocracy emphasizes the role of individual over structural factors in personal achievements, leading to greater legitimization of inequalities (premingerMeritocracyServiceEthnocracy2020?; trumpWhenWhyEconomic2020?; Hadjar 2008; Madeira et al. 2019). Such research has raised criticism of meritocracy as a moral standard of distribution, arguing that it would be an unfulfilled promise given the still preponderant weight of non-meritocratic elements upon individual status and social mobility(sandelTyrannyMeritWhat2020a?; witteveenReconsideringMeritocraticPower2020?; Arrow, Bowles, and Durlauf 2000; Goldthorpe 2003; markovits_Meritocracy_2019?; khan_Privilege_2013?).
Due to the role that meritocratic beliefs play in the justification of individual achievement or failure in contemporary societies Markovits (2019), multiple studies have evaluated the relationship between meritocratic beliefs and personal and/or contextual characteristics. For example, some studies have linked meritocracy to the reinforcement of socio-economic, gender, and ethnic stereotypes (Madeira et al. 2019; girerdNeoliberalismIdeologicalBarrier2020?; premingerMeritocracyServiceEthnocracy2020?), as well as the effects of meritocratic beliefs in educational (generettStoriesWeTell2020?; owensENGINESSOCIALMOBILITY2020?) and organizational contexts (perezAdvancingCareersMerit2020?; aielloNewEvidenceFirmuniversity2019?).
Most of the studies so far have used indicators from existing standard surveys (as the International Social Survey Programme - ISSP) to measure meritocracy. However, as we will show later, the methods of measuring meritocracy vary extensively among studies. In many cases, similar phenomena are associated with different indicators, and conversely, often different phenomena are measured with similar indicators. Such trends not only limits the comparability of studies, but also the ability to understanding the effects of meritocratic beliefs across different fields.
Based on a critical analysis of different approaches to the measurement of meritocracy to date, this article proposes an instrument to both measure and relate two key aspects in the study of meritocracy: perceptions and preferences. Furthermore, as a second axis of analysis, it considers the elaboration of indicators regarding meritocratic and non-meritocratic aspects, demonstrating that they are not two poles of the same continuum as some previous studies would seem to suggest. The measurement system outlined is oriented to generate an instrument as simple and brief as possible, being suitable to be used in public opinion surveys and allowing to integrate meritocratic beliefs in the study of different social phenomena.
The black-box of meritocratic beliefs
In the following we discuss four critical aspects in studies conceptualizing and measuring meritocracy and that are the basis of our scale proposal.
- Conceptual components: Is merit only effort?
One recent definition of meritocracy by Mijs (2019) is the following: “when I discuss meritocracy beliefs, I am referring to citizens’ belief in the importance of hard work relative to structural factors.” (Mijs 2019, pg.9). In the subsequent operationalization, this is associated with the following question and indicator: “how important you think it is for getting ahead in life: (a) hard work,” scored from 1 to 5 on a likert scale. The assumptions behind such definition are worth discussing in light of the conceptual meaning of meritocracy and its possibilities of operationalization.
The item used by Mijs (2019) is part of an items’ battery present in several international surveys, usually called “reasons to get ahead.” This battery displays a series of indicators related to what people consider important to get ahead in life: hard work, education, ambition, a wealthy family, the right connections, religion, race, and gender. Therefore, when considering only one of the items, it means that other aspects that could be associated with talent, such as education, would not be deemed as meritocratic. As he points out: “Hard work is arguably the most meritocratic part of Michael Young’s equation: ‘Merit = Intelligence + Effort,’ for the simple fact that intelligence itself is influenced by a non-meritocratic factor: who your parents happen to be” (p.5).
In this measure of meritocratic beliefs, we can observe a couple of strong conceptual assumptions: effort would not depend on parental influence, and talent (as innate ability) is not meritocratic (contrary to Michael Youngs’ original conceptualization). This conceptual and measurement-based assumption is found in other studies, that also assume effort to be the main and only aspect of meritocracy (girerdNeoliberalismIdeologicalBarrier2020a?; bubakPerceptionsMeritocracyNote2019?). Is effort the only dimension behind the concept of meritocracy, or includes other components, such as talent (as in its original conceptualization). The question of whether talent is or is not considered meritocratic is certainly an interesting topic to discuss from a philosophical point of view, but for those working on empirical studies it should be something that face empirical scrutiny. Besides, the consideration of talent as part of meritocracy certainly opens some interesting avenues of research, for instance in studies showing that for the elites meritocracy is more related to talent, whereas effort is something more characteristic of meritocracy of the middle and low classes ([citar Atria ABS]). Actually, in a posterior paper by (mijsHowInformationEconomic2020?), somewhat contradictory with its previous definition, he considered talent as an indicator of meritocracy.
b. Beliefs?
Several approaches to the empirical study of meritocracy based on public opinion surveys, refer to the concept of beliefs, however different meanings and operationalizations usually underlie this concept. To illustrate this point, lets consider the most commonly used items for operationalizing meritocratic beliefs in survey research (as ISSP), which is the “reasons to get ahead” battery. It consists in a series of items asking “how important you think it is for getting ahead in life” and then lists several factors, such as effort, education, parental wealth, and contacts. Another version of this same battery used in several surveys - sometimes along the previous one - asks about “how important you think it should be …,” then listing the same concepts. Therefore, the question raised here is: Which one of both is a “belief?” What is or what it should be?
The term belief has an ambiguous character in the literature, conceived as “idea-elements” by (converseNatureBeliefSystems1964?) or “considerations” by (zallerNatureOriginsMass1992?). As (kluegelBeliefsInequalityAmericans2017?) pointed out about the scope of beliefs: “This usage encompasses such more specific social-psychological concepts as values, perceptions, and attitudes” (p.30). Therefore, beliefs cover almost anything related to subjective factors. To this regard, a relevant distinction in the field of inequality beliefs was made by (janmaatSubjectiveInequalityReview2013?): “Perceptions refer to subjective estimates of existing inequality (i.e. thoughts about what is). Beliefs are here defined as normative ideas about just inequality (i.e. thoughts about what should be)”(p.359). Several papers dealing with meritocracy use the term beliefs (i.e. what should be), while actually referring to perceptions (i.e. what is). This occurs not only in Mijs (2019), but also in Reynolds and Xian (2014), in which the term beliefs is used to talk about perceptions, whereas other authors use general terms such as attitudes (Kunovich and Slomczynski 2007). The first attempt to shed light on this issue was made by (duru-bellat_whos_2012?), who used the question “how important should the number of years spent in education and training be in deciding how much money people ought to earn?” as a proxy for “desired” meritocracy (beliefs). They then determined “perceived” meritocracy, using the questions: “Would you say that in your country, people are rewarded for their efforts?” and “… people are rewarded for their skills?”
Is the belief in meritocracy a perception or a preference with normative meaning? In order to expand the analytical conceptual framework, we believe that both dimensions should be included in analyses, as proposed by (duru-bellat_whos_2012?). This opens up the possibility of analyzing whether perceptions and preferences are actually related (i.e. have a high correlation), or whether they are independent aspects of the same phenomenon. As (son_hing_merit_2011-1?) have pointed out, “People can believe that outcomes ought to be distributed on the basis of merit and yet vary in their perceptions of whether this is how society currently operates” (p. 435). In other words, normative beliefs should be considered while taking perceptions into account: a strong normative belief in meritocracy may mean something totally different to someone perceiving high meritocracy, when compared to someone perceiving low meritocracy. To avoid the confusion generated by the term “belief,” we propose the terms meritocratic preferences (“what should be”), and meritocratic perceptions (“what is”), as they better reflect the two facets of meritocracy under scrutiny (Castillo et al. 2019).
c. Non-meritocratic aspects
A number of studies have dealt with aspects opposed to meritocracy, as for instance the use of personal advantages (as contacts or having a wealthy family) to get ahead in life. For instance, Kunovich and Slomczynski (2007) used an items’ battery listing a number of factors in relation to “How important each should be in deciding pay…” (as (duru-bellat_whos_2012?) for desired meritocracy). They consider factors such as education and responsibility as meritocratic, giving them a value of 1 if considered “essential” in the scale response, whereas factors such as having a family or children are valued 1 when rated as “not important at all” (i.e. reverse coded). The assumption behind this approach is that rejecting a supposed non-meritocratic aspect (as having family and/or children) implies a stronger belief in meritocracy. A similar approach of reverse-coding non-meritocratic items was taken by Newman, Johnston, and Lown (2015), using the same principles applied in the “Preference for the Merit Principle Scale” (Davey et al. 1999).
The assumption that meritocratic and non-meritocratic elements are poles of the same continuum was tested by Reynolds and Xian (2014) using the “get ahead” perceptions’ battery items mentioned above. They considered education, ambition, and hard work as meritocratic, and other factors such as family wealth and connections as non-meritocratic. Despite making and proving this distinction, however, the authors end up subtracting one dimension from the other, thus coming back to the assumption that they are two poles of the same continuum as Kunovich and Slomczynski (2007) did. Similarly, (roex_attitudes_2019?) used ISSP indicators for perceived meritocracy and non-meritocracy to build a single score by reverse coding the non-meritocratic items. Therefore, the treatment of non-meritocratic items has been rather inconsistent across studies and the assumption that they are the simple opposite of meritocracy requires further empirical assessment.
d. Accounting for measurement error
Finally, most studies in meritocracy so far have not accounted for the issue of measurement error (brownBookReviewTimothy2010?; bollenStructuralEquationsLatent1989?) (Ansolabehere, S., Rodden, J., & Snyder Jr, J. M., 2008), as they use single indicators and/or simple average indexes for measuring meritocracy. Such strategy assumes that the latent construct is measured perfectly (i.e. no error or residual variance) by the indicators chosen, going as far as to propose that “… In choosing this strategy of index construction, we argue that support for meritocracy is not a latent variable” (Kunovich and Slomczynski 2007, 653–54). Some advances were made by Reynolds and Xian (2014) through conducting a principal component analysis of meritocratic and non-meritocratic dimensions, but somewhat contradictorily, they chose a sum index despite proving a multidimensional latent structure.
An instrument proposal
Based on the previous assumptions and limitations of the measurements of meritocracy, we propose and test an instrument with the following characteristics:
Multidimensionality, incorporating previous distinctions between preferences and perceptions, as well as between meritocratic and non-meritocratic aspects.
Multiple indicators for each dimension, in order to account for measurement error in a confirmatory factor analysis context.
Based on previous indicators as much as possible, for the sake of keeping comparability between studies.
Brief, as to be used in regular public opinion surveys. In this respect, it differs from the proposal of “Preference for the Merit Principle Scale” (Davey et al. 1999), as they use 15 items for just one dimension (aside from the problem of reverse-coding non-meritocratic items).
The proposed measurement framework is depicted in Figure 1:
Figure 1: Model of perception and preferences for meritocracy and non-meritocracy
The columns “Perceptions” and “Preferences” represent the distinction between these two concepts, usually confused under the label “beliefs” (Castillo et al. 2019). Perceptions refer to the extent to which people observe that meritocracy works or applies in their society, which relates to items such as “I think hard work is important to get ahead in society.” Preferences refer to normative expectations that are usually linked to a “should” expression (e.g. whether hard work should be related to payment). The rows in Figure 1 consider the distinction between meritocratic and non-meritocratic dimensions (Reynolds and Xian 2014). These aspects have usually been treated as different ends of the same continuum in previous research, an assumption that requires empirical scrutiny. Non-meritocratic elements usually refer to the use of personal contacts or family advantages to get ahead in life.
Regarding the selection of indicators, most are taken or adapted from previous studies for the sake of comparability. For meritocratic indicators, we use effort and talent as the main components of the traditional concept of merit as defined by M. Young (1962), whereas for non-meritocratic dimensions, we use wealthy parents and good contacts. Descriptions of the specific items are outlined in the methodology section.
The research hypotheses behind this conceptualization and measurement model are as follows:
\(H_1\). The perception of meritocracy is a latent variable based on indicators of the importance attributed to talent and effort to get ahead in life.
\(H_2\). The perception of non-meritocracy is a latent variable that derives from indicators related to agreement with the statement that people with contacts and rich parents manage to get ahead.
\(H_3\). Meritocratic preferences are a latent variable based on the normative value of effort and talent.
\(H_4\). Non-meritocratic preferences are a latent variable based on the normative value of the use of personal contacts and having wealthy parents.
Methodology
The analysis is organized in three studies. Study 1 serves as a background to our instrument proposal by attempting to operationalize the meritocracy dimensions with available data. For this we use data from the International Social Survey Programme (ISSP). Although we are aware that the available indicators in this dataset do not allow a precise operationalization of the conceptual model, we know that this dataset is the most widely used in comparative meritocracy studies (as well as overall subjective inequality) and therefore it is certainly useful to look at its potentials and limitations to this regard. Study 2 is the core of this paper and consists of the proposal of a new scale for measuring meritocratic perceptions and preferences. Finally, in Study 3 we perform additional validity analysis of the meritocracy scale.
Study 1
Data
The data used in the analysis corresponds to the last available wave of the social inequality module from the International Social Survey Programm (ISSP), which is the most specialized international comparative survey in perceptions, attitudes and beliefs about inequality related issues. This wave corresponds to the year 2009 and covers attitudes towards a series of topics dealing with social inequality across 26 countries and XXXX individuals. Although there is data available from this module for previous waves (1987, 1992 and 1999), unfortunately there are several variables that are important for this research that were not included, particularly in 1987 and 1999, reason why we only use the last wave.
Variables
There are a series of indicators in the ISSP survey that in the following we attempt to classify in our meritocracy measurement scheme. However, it is important to mention that the items were not originally thought as a specific measure of meritocracy. Still, many of the items have been used in previous research on meritocracy (and certainly will keep being used in the future) and before proposing a new measure we consider to give a first test to our proposed conceptualization using available data that not only covers several countries but also is validated by the research community.
The variables are presented below organized in perceptions and preferences:
Perception of meritocracy/non-meritocracy: for operationalizing perceptions the closest set of ISSP’s indicators comes from the question asking about perceptions for opportunities to get ahead, which are usually considered as “meritocratic beliefs” in previous research. The general heading of the battery is: “To begin we have some questions about opportunities for getting ahead. Please tick one box for each of these to show how important you think it is for getting ahead in life.” This is followed by a list of statements to be answered in a scale with the following levels numbered from 1 to 5: essential, very important, fairly important, not very important, not important at all. Below the items are listed classified according to the meritocracy/non-meritocracy character:
- Meritocratic perception:
- How important is having ambition?
- How important is hard work?
- Non-meritocratic perception
- How important is coming from a wealthy family?
- How important is having well-educated parents
- How important is knowing the right people
- How important is having political connections
- How important is a person’s race?
- How important is being born a man or a woman?
- Meritocratic perception:
The classification is based on criteria of internal motivation (meritocratic) and structural constraints (non-meritocratic). There were two items from the battery that were excluded from the analysis as they would not fit into the classification. The first one was “having good education yourself,” since it was not clear whether this is could be due to individual motivation or system opportunities, and the second was “giving bribes,” as introduced elements of criminality that were beyond a non-meritocratic perception.
Preferences for meritocracy-non meritocracy: for operationalizing normative preferences we used a list of items related to reasons for pay. The ISSP question was: In deciding how much people ought to earn, how important should each of these things be, in your opinion, rated in the same essential-non important at all scale as the questions for meritocratic preferences. The reasons are:
- Meritocratic preferences
- How well he or she does the job?
- How hard he or she works at the job?
- Non-meritocratic preferences
- What is needed to support a family?
- Whether the person has children to support?
- Meritocratic preferences
Table 1
| Mean | SD | Min | Max | |
|---|---|---|---|---|
| Importance: hard work | 3.89 | 0.93 | 1 | 5 |
| Importance: having ambition | 4.00 | 0.88 | 1 | 5 |
| Importance: wealthy family | 2.91 | 1.17 | 1 | 5 |
| Importance: educated parents | 3.24 | 1.06 | 1 | 5 |
| Importance: race | 3.56 | 1.01 | 1 | 5 |
| Importance: gender | 2.66 | 1.20 | 1 | 5 |
| Importance: knowing people | 2.21 | 1.15 | 1 | 5 |
| Importance: political connections | 2.21 | 1.16 | 1 | 5 |
| Reasons for pay: well job | 4.11 | 0.75 | 1 | 5 |
| Reasons for pay: hard job | 4.02 | 0.79 | 1 | 5 |
| Reasons for pay: support family | 3.48 | 1.04 | 1 | 5 |
| Reasons for pay: has children | 3.30 | 1.12 | 1 | 5 |
Methods
The estimation was performed using Confirmatory Factor Analysis (CFA). CFA was conducted using the lavaan R package (version 0.6-3; Rosseel, 2020), with diagonally weighted least squares (DWLS) estimation due to the items’ ordinal level of measurement (Kline, 2016; Rosseel, 2020). As recommended by Brown (2008), we assessed model fit by jointly considering the comparative fit index and Tucker-Lewis Index (CFI and TLI; acceptable fit > 0.95), Root of the average squared residual approximation (RMSEA; acceptable fit < 0.08), Chi-square: (p-value; acceptable fit > 0.05, and Chi-square ratio > 3).
Results
Figure 2
Figure 2: Descriptive plot
Table 2
| Meritocratic | Non-meritocratic | Meritocratic | Non-meritocratic | |
|---|---|---|---|---|
| How important is having ambition? | 0.71 | |||
| How important is hard work? | 0.73 | |||
| How important is coming from a wealthy family? | 0.69 | |||
| How important is having welleducated parents | 0.61 | |||
| How important is knowing the right people | 0.62 | |||
| How important is having political connections | 0.59 | |||
| How important is a person’s race? | 0.61 | |||
| How important is being born a man or a woman? | 0.64 | |||
| How well he or she does the job? | 0.76 | |||
| How hard he or she works at the job? | 0.91 | |||
| What is needed to support a family? | 0.84 | |||
| Whether the person has children to support? | 0.92 | |||
| \(\chi^2\text{(df)}\) | 21308.5(48) | |||
| \(\text{CFI}\) | 0.959 | |||
| \(\text{TLI}\) | 0.944 | |||
| \(\text{RMSEA}\) | 0.098 | |||
| \(N\) | 46594 | |||
| Note: Standardized factor loadings using DWLS estimator ; CFI = Comparative fit index ;TLI = Tucker-Lewis index; RMSEA = Root mean square error of approximation |
Study 2
Data collection
The data was obtained through an online questionnaire, which was part of a larger study on meritocracy and preferences developed in Chile in 2019 funded by the national scientific agency ANID. The questionnaire was programmed in Qualtrics and the fieldwork was conducted by an external online survey agency (netquest.cl) between December 2019 and January 2020. The sample was selected from a non-probabilistic quota design in three large cities in Chile (Santiago, Concepción & Antofagasta). The quotas for gender, age, and education level were generated based on a survey by the Public Studies Center (cepEncuestaCEPMayo2019?), which is a well regarded counterpart agency of the ISSP (International Social Survey Programme) in Chile. A total sample of 2,141 people was collected, excluding those who did not answer the questions on the scale, and those who did not accept informed consent. There were no significant differences between our sample and the wider population for most socio-demographic characteristics, with the exception of educational level (see Table 10 in appendix). As is often the case with online surveys, there were some limitations in achieving the quotas for lower educational levels [citas - LUCHO].
Instrument design
The proposed scale of perceptions and preferences relating to meritocracy consisted of eight indicators that were grouped into the four dimensions listed earlier: Perceptions (meritocratic/non-meritocratic) and preferences (meritocratic/non-meritocratic). In order to achieve at least some comparability with previous studies, the questions were adapted from the items battery “reasons to get ahead” (ISSP/GSS), which has been widely used for operationalizing meritocracy in previous studies (Mijs 2019; duru-bellat_whos_2012?; Reynolds and Xian 2014). The aforementioned eight items, ordered according to dimensions, are presented in Table 3. These eight likert-type items have five response alternatives, ranging from “Completely disagree”(1) to “Completely agree” (5).
| Dimensions | Factor | Statement (english) | Statement (spanish) |
|---|---|---|---|
| Perception | Meritocratic | Those who try harder get greater rewards than those who work less. | Quienes más se esfuerzan logran obtener mayores recompensas que quienes se esfuerzan menos. |
| Those who have more talent achieve greater rewards than those who have less talent. | Quienes poseen más talento logran obtener mayores recompensas que quienes poseen menos talento. | ||
| Non meritocratic | Those who have rich parents succeed. | Quienes tienen padres ricos logran salir adelante. | |
| Those who have good contacts succeed. | Quienes tienen buenos contactos logran salir adelante. | ||
| Preference | Meritocratic | Those who try harder should get greater rewards than those who work less. | Quienes más se esfuerzan deberían obtener mayores recompensas que quienes se esfuerzan menos. |
| Those who have more talent should get greater rewards than those who have less talent. | Quienes poseen más talento deberían obtener mayores recompensas que quienes poseen menos talento. | ||
| Non meritocratic | It’s fine that those with rich parents get ahead. | Está bien que quienes tienen padres ricos salgan adelante. | |
| It’s fine that those who have good contacts get ahead. | Está bien que quienes tienen buenos contactos salgan adelante. |
Administration sets
With the objective of evaluating the effect of indicator ordering in the responses, three different versions of items’ order were designed and randomly assigned, as depicted in Figure 3. The scale was presented to the first group (n = 712) in the order that appears in Table 3 according to perceptions and preferences. For the second group (n = 717), the order was reorganized according to perceptions and preferences over the same topic, e.g. for the topic of hard work, the item about perception was followed by the item about preference, and the same for the rest of the topics. Finally, for the third group (n = 712), the items were completely randomized.
Figure 3: Survey flow
Methods
To test the scale’s underlying constructs, we employed confirmatory factor analysis models (CFA). The models estimated one factor for each of the proposed dimensions, as represented in the following figure:
Figure 4: Theoretical model
As in Study 1, CFA was conducted using the lavaan R package (version 0.6-3; Rosseel, 2020), with diagonally weighted least squares (DWLS) estimation due to the items’ ordinal level of measurement (Kline, 2016; Rosseel, 2020). The fit indexes and cut-off criteria were the same as in Study 1.
A pre-registration was made in the OSF platform, available at the following link: https://osf.io/z45y2. Included in this pre-registration are the hypotheses regarding the four-dimensional conceptual model underlying the scale, the variable measurement levels, the statistical tests to be performed with their respective evaluation parameters, and other important aspects of the research design.
Results
Descriptive analyses
As can be seen in the Table 4, the indicators have values ranging from 1 (totally disagree) to 5 (totally agree). Averages are observed from 2.41, corresponding to preference-contacts; to 3.89, corresponding to preference-effort. Both indicators are consistent with a general adherence to meritocracy as reported in previous studies, privileging individual aspects such as effort (bucca_merit_2016?; frei_fruto_2020?).
| Mean | SD | Min | Max | |
|---|---|---|---|---|
| A. Perception Effort | 3.20 | 1.38 | 1 | 5 |
| B. Perception Talent | 3.02 | 1.16 | 1 | 5 |
| C. Perception rich parents | 3.66 | 1.36 | 1 | 5 |
| D. Perception contacts | 3.79 | 1.24 | 1 | 5 |
| E. Preferences Effort | 3.89 | 1.25 | 1 | 5 |
| F. Preferences Talent | 3.24 | 1.19 | 1 | 5 |
| G. Preferences rich parents | 2.69 | 1.18 | 1 | 5 |
| H. Preferences contacts | 2.41 | 1.11 | 1 | 5 |
The graphs presented in Figure 5 display disaggregated and comparable information of the different response categories for each item. As it can be observed, in general there is more agreement in the perception of non-meritocratic items than in meritocratic items, while in the case of preferences, the opposite occurs. As far as preferences are concerned, the preponderant role of effort over talent as a criterion of meritocratic preference is noteworthy.
Figure 5: Descriptive plot
In Figure 6, moderate to high intensity relationships are observed between indicators that correspond to the same factor (e.g., perception of meritocracy by effort and by talent, r=0.56). The relationships between meritocratic perceptions and preferences are also salient, which is not the case for non-meritocratic indicators.
Figure 6: Polychoric correlation plot
In sum, descriptive analyses point to a relative adherence to meritocracy, which is expressed as a greater preference for meritocratic criteria, and a lesser preference for non-meritocratic criteria, even when the preference for non-meritocratic elements reaches a proportion of more than 20%. Likewise, a relatively low perception of meritocracy is observable. In addition, we observed a coherent relationship between the indicators of the same dimension as proposed in the pre-registration of the study; that is, the pairs of items associated with a specific factor show correlations with a large effect size. In particular, the associations between effort and talent are relevant, challenging previous assumptions that talent would not be a meritocratic criterion (Mijs 2019), otherwise the correlation would be zero or negative. In addition, we observe that there is no negative correlation between meritocratic and non-meritocratic aspects, undermining the assumptions of previous studies that suggested that these dimensions would be the opposite poles of one same continuum (Reynolds and Xian 2014).
Confirmatory Factor Analysis
This section estimates the fit of the conceptual model behind the Perceptions and Preferences for Meritocracy Scale. For this, we first estimate a confirmatory factor analysis model for the whole sample, and secondly we test the order effects applying the same model to each of the three order permutations.
Figure 7 shows the results of the estimation for the four-factor model using the complete sample. This model shows adequate fit indicators (CFI = 0.977, RMSEA = 0.079, \(\chi2\)(df = 14) = 28,03, p = 0.014) and it presents factorial loads with weights that vary between 0.63 and 0.88.
Figure 7: Confirmatory factor analysis of the Perceptions and Preferences for Meritocracy Scale
Attending to the correlations between the latent variables as depicted in Figure 8, it is observed that the variables with the strongest correlations are meritocratic perceptions and preferences, whit an \(r=0.59\), while non-meritocratic perceptions and preferences show a correlation close to 0. Besides, and as commented before in the analysis of the single items correlations, the latent variables of meritocratic and non-meritocratic versions of both perceptions and preferences show very low correlations. This last finding gives further evidence regarding the lack of unidimensionality of meritocratic and non-meritocratic aspects: they do not appear as the opposite poles of the same continuum and based on this evidence the use of reverse-coding for non-meritocratic items as indicators of meritocracy should be avoided.
Figure 8: Latent variables correlation
In a second step we test the effect of the items’ order in the confirmatory model. Table 5 shows the fit indicators of the models estimated for each of the three versions of the items’ order described in the methodology, being the first one according to perceptions and preferences, the second one according to meritocratic and non-meritocratic aspects, and the third one completely randomized. Regardless of the group and order of items, all models obtained an adquate fit, with CFI’s above 0.95 and RMSEA’s below 0.08. However, none of the models achieved a non-significant chi-square, something expected in large samples as the one used here). Model 1 shows the fit indicators for the estimation of the CFA model to the first version order (perceptions-preferences), which was the one obtaining best fit (CFI=0.993, TLI=0.995, RMSEA=0.034, \(\chi2\)(df=14)=42.276), whereas Model 2 for the fixed order according to merit/non-merit items shows the comparatively worst indicators. The CFA fit indices for the completely randomized items’ order (Model 3) is somehow an intermediate fit indexes when compared with the previous two. Still, it keeps all the indicators within the acceptable cut-off criteria and besides it controls for possible order effects in the administration of the instrument.
| Model | \(N\) | Estimator | \(\chi^2\) | df | CFI | TLI | RMSEA |
|---|---|---|---|---|---|---|---|
| Model 1 | 712 | DWLS | 25.631 | 14 | 0.998 | 0.996 | 0.034 |
| Model 2 | 717 | DWLS | 67.652 | 14 | 0.984 | 0.967 | 0.073 |
| Model 3 | 712 | DWLS | 41.633 | 14 | 0.991 | 0.981 | 0.053 |
| 1 Model 1: fixed order by perception/preference | |||||||
| 2 Model 2: fixed order by merit-non merit | |||||||
| 3 Model 3: Randomized order |
Additional validity analyses
We performed two further analyses in order to add evidence regarding the quality of the scale. Firstly we analyzed the convergent validity by exploring the association of the merit scale with related concepts and itscorrespondent measurements. Secondly, we test the internal consistency of the measurement model behind the scale comparing with a different sample through measurement invariance procedures .
Convergent validity
Data
For this analysis we examined data from one wave of the same study described above, which was a three-wave panel survey. The last wave of the study included additional measures that allow testing the correlations of the merit scale with some related constructs as opportunity beliefs and personal wherewithal. After listwise deletion of missing cases in socio-demographics information, a total of 1422 individuals took part in the third wave: 668 (46.97%) women and 754 (53.03%) men. The ages of 50.57% of them are 45 years or older, and 36.28% held a tertiary degree.
Intruments
The following scales were included for testing their correlations with the the 8-item meritocracy scale:
Opportunity beliefs: We used two items of the social inequality module of the International Social Survey Programme. Studies use generally both indicators to measure meritocratic and non-meritocratic beliefs (McCall2017Exposurerisinginequality?; Mijs 2019). One item measures the importance of individual factors in determining life outcomes by asking for the importance of hard work for getting ahead in life (M=3.76, SD=0.93). The second item asks for the importance of coming from a wealthy family and captures the importance of structural factors in getting ahead (M=2.75, SD=1.27). Participants responded to each item using a 1 (not important at all) to 5 (essential) response scale. We call the first item and the second one .
Personal wherewithal: the questionnaire included The Neoliberal Beliefs Inventory (NBI), which considers four factors: Government Interference preferences, Competition preferences, System Inequality perception and Personal Wherewithal (bay-chengTrackingHomoOeconomicus2015?). We used the factor personal wherewithal that reflects meritocracy beliefs in terms of the importance of personal attributes as strength and skills to yield success. The construct consists of 8 items (e.g., “Any goal can be achieved with enough hard work and talent,” “I’ve benefited from working hard, so there’s no reason others can’t,” “Anyone who is willing to work hard can be succesful in Chile”). Participants answered the items using a 1 (totally disagree) to 6 (totally agree) scale. We simple computed mean scores, with higher scores indicating stronger support for meritocracy (M=3.32, SD=0.88, \(\alpha\) = 0.89).
Both instruments cover mainly the perceptual side of the concept. Therefore, in general we expect larger correlation with meritocratic perceptions than with preferences. In the case of opportunity beliefs, the hard work item should correlate positively with meritocratic perception whereas the importance for coming from a wealthy family is expected to correlate positively with non-meritocratic perceptions. Regarding the second instrument of personal wherewithal, we also anticipate a positive correlation with meritocratic perception and low to null correlations with the other dimensions of the merit scale.
Results
We used polyserial correlations to evaluate relationships of meritocractic and non-meritocratic preferences and perceptions with the items of the opportunity beliefs battery. For personal wherewithal, we examined Pearson correlations.
Table 6 shows the polyserial and pearson correlations between our meritocracy scale and common measures of meritocracy. The findings provide evidence for convergent validity for the scale. As we expected, preferences show very weak correlations. Findings also indicate that correlations of perceptions with opportunity beliefs are consistent with our expectations. The item for social origin shows a positive and moderate association with unmeritocratic perception (r=.354), and a negative and low correlation with meritocratic perception (-0.197). Regarding the item of hard work, the correlation with meritocratic perception is positive but weak (r=.187). One explanation for this result is that the support for hard work is very high among respondents—only 8.6 percent indicates not very important or not important at all.
| Social Origin | Hard Work | Wherewithal | |
|---|---|---|---|
| Meritocratic Perception | -0.1975 | 0.1877 | 0.3921 |
| Unmeritocratic Perception | 0.3547 | -0.0775 | -0.1844 |
| Meritocratic Preference | 0.1078 | 0.0779 | 0.0819 |
| Unmeritocratic Preference | -0.0397 | 0.0824 | 0.2082 |
The measure of NBI for personal wherewithal also provides evidence for the convergent validity of the merit scale. Results indicate a positive correlation of this measure with meritocratic preference (r=.392), suggesting that both variables capture similar concepts. The NBI’s factor depicts a positive association with unmeritocratic preference (r=.208). This finding is consistent with literature, inasmuch as the belief for meritocray is associated with the justification of inequality (McCall2017Exposurerisinginequality?; Madeira et al. 2019). Furthermore, results of our confirmatory factor analysis indicate that meritocratic perceptions and unmeritocratic preferences show a positive and weak correlation.
Measurement invariance
The modeling for invariance testing has been gaining more relevance in sociological survey studies, mainly due to the interest in the validity of measurement scales in comparative studies in various social and cultural contexts (Davidov et al. 2014). In this section we will use an analysis of invariance between groups, where the proposed factorial structure is expected to be independent of belonging to the sample corresponding to the original study, comparing it with another more recent study available and where the meritocracy scale was also incorporated.
The invariance measurement procedure consists of a series of nested models to which restrictions are progressively incorporated on the parameters of the measurement model. The literature generally suggests that this modeling should be done through four levels or types of progressive restriction (Milfont and Fischer 2010; Millsap 2011; van de Schoot, Lugtig, and Hox 2012):
Configural: the model is estimated only indicating the factorial structure used in the CFA.
Weak: an equality restriction is applied to the factor loadings in the different groups, that is, the loadings are forced to be identical in both measurements.
Strong: equality restrictions are added to the intercepts of each indicator.
Strict: equality restrictions are added to the error variances of each indicator.
Data
For testing the invariance we compared the data described previously with a new data source coming from an online survey carried out during the first half of 2020. The characteristics of this sample in terms of application and coverage are equivalent to those of the previous study. The final sample obtained contains 1,242 cases, where 605 (48.71%) are women and 637 (51.28%) are men. 48.84% of them are 45 years of age or older, and 33.97% have tertiary education or higher.
Instruments
The items of the meritocracy scale were identical to the original. The items were administrated in a randomized order (which corresponds to the application modality of Group 3 present in Figure @ref(fig: appmod)).
Results
Tabla 9 shows de results of the CFA estimation of the meritocracy scale in this new sample. As observed, the loadings and the fit indices show similar results as the ones presented for the original study above, which gives a first base with wich to start the invariance analysis.
The first step for invariance testing is the estimation of the configural model, which serves as the baseline for further comparison and it is expected to adequately meet the global fit criteria of a measurement model. Although the chi-square statistic is used as a global measure of fit,since its sensitivity to sample size it is recommended to use three additional fit indices: Comparative Fit Index (CFI), which should have a value greater than 0.95; Root Mean Square Error Approximation (RMSEA), which must be in the range of values of 0.05 and 0.08; and the Standardized Root Mean Square Residuals (SRMR) which must be less than 0.08 (van de Schoot, Lugtig, and Hox 2012). Besides, the literature on measurement invariance suggests some complementary approaches for the evaluation of the fit, from which we will consider the incremental adjustment of the fit indexes (Cheung and Rensvold 2002; Milfont and Fischer 2010; Dimitrov 2010) and the ANOVA test for means comparison in nested models (Newsom 2015).
| Model | \(\chi^2 (\text{df})\) | CFI | RMSEA (90 CI) | \(\Delta \chi^2 (\Delta \text{df}\)) | \(\Delta \text{CFI}\) | \(\Delta \text{RMSEA}\) | Decision |
|---|---|---|---|---|---|---|---|
| Configural | 316.27 (28) | 0.944 | 0.078 (0.07-0.086) | ||||
| Weak | 323.62 (32) | 0.943 | 0.073 (0.066-0.081) | 7.347 (4) | -0.001 | -0.005 | Accept |
| Strong | 330.66 (36) | 0.942 | 0.07 (0.063-0.077) | 7.045 (4) | -0.001 | -0.004 | Accept |
| Strict | 426.74 (44) | 0.925 | 0.072 (0.066-0.078) | 96.08 (8) *** | -0.017 | 0.002 | Reject |
| Note: N = 3383; Group 1, n = 1242; Group 2, n = 2141, ***p < 0.001 |
Table 7 shows the results of the measurement invariance estimation. When attending to the traditional invariance test of \(\Delta \chi^2 (\Delta \text{df})\), the results support the invariance at the strong level meaning that the fit of the factor model of the merit scale is equivalent across samples when constraining factor loadings and intercepts to being equal. Such result is considered in general as evidence of invariance [citar paper Fischer en SJR], as strict forms of measurement invariance rarely hold [citar van der Schoot et al]. Still, the comparability of latents means requires strict invariance which in this case does not hold when considering \(\Delta \chi^2 (\Delta \text{df})\). Nevertheless, the criteria of \(\Delta \text{CFI}\) used for comparing models is close to the rejection criteria of >.01, whereas the \(\Delta \text{RMSEA}\) fulfills the requirements of being below of the cut-off criteria as suggested by (chen?). Therefore, using this last standard the level for strict invariance holds for the meritocracy scale.
Conclusions
Studies that attempt to characterize and compare societies by their support for meritocratic beliefs have used different approaches. As most studies use secondary survey data, they tend to assume that the available indicators represent an underlying meritocratic construct. A review of these studies reveals several non-tested assumptions, as well as the use of similar indicators to represent different constructs and dimensions of meritocracy. As the existence of heterogeneous approaches certainly has consequences for the advancement of the study of meritocracy, this paper presented a comprehensive conceptual framework for the empirical study of meritocracy, building upon previous research. This framework was then tested against new survey data.
We identified four critical aspects regarding the measurement of meritocracy in previous studies: unidimensionality, the ambiguous use of the term “beliefs,” the use of non-meritocratic indicators as opposed to meritocratic indicators, and the consideration of measurement error. The proposed 8-item scale, “Perceptions and Preferences for Meritocracy,” was designed and tested in order to deal with these four issues. Our results indicate that perceptions and preferences seem to be two related but different dimensions, often confounded in previous research under the label of “beliefs.” Meritocratic and non-meritocratic dimensions do not appear to constitute poles of the same continuum, as some previous studies have assumed. Regarding the possible effects of items order in the estimation of latent variable, we tested three different order versions and found evidence that suggest the use of the randomized version of the scale. Furthermore, we found evidence of convergent validity as well as measurement invariance that approach well the requirement of comparability between samples.
The four-dimensional framework and structure of the Perceptions and Preferences for Meritocracy scale opens several avenues for future research. For instance, distinguishing perceptions from preferences will allow us to evaluate the extent to which different societies are accustomed to, or satisfied with, the perceived level of meritocracy, in terms of differences between what is perceived and what is preferred. Additionally, given that non-meritocratic factors are not necessarily related to meritocratic ones, our framework makes it possible to assess the perceived legitimacy of practices such as the use of personal contacts and their interference (or not) with meritocratic ideals in different societies. The consideration of structural factors, both individual and societal, will allow us to advance future hypotheses relating to meritocratic legitimacy to a more comprehensive level than the current one, which is based solely on meritocratic perceptions. Furthermore, the impacts of different configurations of the four-dimensional framework on practices and behaviors such as corruption, civic involvement, and political alignment, is an area that requires additional research. Such future agendas could be specially relevant in times of economic crisis and growing inequalities, that could entail changes in the legitimation of the current distributive structure based on meritocratic ideals.
References
Appendix
| Variables | 1 | 2 | 3 | 4 | 1 | 2 | 3 | 4 | 1 | 2 | 3 | 4 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A. Who the more they try they manage to get bigger rewards that those who striveless. | 0.69 | 0.76 | 0.70 | |||||||||
| Perception Talent | 0.81 | 0.72 | 0.65 | |||||||||
| Perception rich parents | 0.85 | 0.84 | 0.81 | |||||||||
| Perception contacts | 0.94 | 0.81 | 0.89 | |||||||||
| Preferences Effort | 0.85 | 0.82 | 0.66 | |||||||||
| Preferences Talent | 0.64 | 0.65 | 0.59 | |||||||||
| Preferences rich parents | 0.55 | 1.04 | 0.78 | |||||||||
| Preferences contacts | 1.26 | 0.52 | 0.77 | |||||||||
| \(\chi^2\text{(df)}\) | 25.6(14) | 67.7(14) | 41.6(14) | |||||||||
| \(\text{CFI}\) | 0.998 | 0.984 | 0.991 | |||||||||
| \(\text{TLI}\) | 0.996 | 0.967 | 0.981 | |||||||||
| \(\text{RMSEA}\) | 0.034 | 0.073 | 0.053 | |||||||||
| \(N\) | 712 | 717 | 712 | |||||||||
| Note: | ||||||||||||
| Standardized factor loadings using DWLS estimator ; CFI = Comparative fit index (scaled); RMSEA = Root mean square error of approximation (scaled) |
| Variables | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| A. Who the more they try they manage to get bigger rewards that those who striveless. | 0.64 | |||
| B. Who possess more talent they manage to obtain greater rewards than those who possess less talent. | 0.75 | |||
| C. Who they have rich parents manage to get out ahead. | 0.69 | |||
| D. Who they have good contacts they manage to get out ahead. | 0.80 | |||
| E. Who the more they try they should get greater rewards than those who they try less. | 0.75 | |||
| F. Who possess more talent they should get greater rewards than those who possess less talent. | 0.60 | |||
| G. It’s fine that those who have rich parents get ahead | 0.65 | |||
| H. Is well that those who have good contacts get ahead. | 0.77 | |||
| \(\chi^2\text{(df)}\) | 104.6(14) | |||
| \(\text{CFI}\) | 0.970 | |||
| \(\text{TLI}\) | 0.941 | |||
| \(\text{RMSEA}\) | 0.072 | |||
| \(N\) | 1242 | |||
| Note: Standardized factor loadings using DWLS estimator ; CFI = Comparative fit index (scaled); RMSEA = Root mean square error of approximation (scaled) |
| Sample | CEP | |
|---|---|---|
| Gender | ||
| Men | 49,82% | 50,52% |
| Women | 50.18% | 49,47% |
| Age | ||
| 18 - 24 | 18,55% | 18,17% |
| 25 - 34 | 18,86% | 17,48% |
| 35 - 44 | 19.09% | 19,98% |
| 45 - 54 | 17,96% | 19,23% |
| 55 - or more | 25,54% | 25.11% |
| Education | ||
| Primary or less | 2,93% | 15,88% |
| High school | 43,23% | 37,04% |
| Non university | 32,63% | 28,93% |
| university or more | 21,21% | 18,13% |